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ABSTRACT. Many educational institutions are shifting their teaching and learning towards 
equipping students with knowledge, skills, and dispositions that prepare them for lifelong 
learning, in a complex and uncertain world. These have been termed "21 s century 
competencies." Learning analytics (LA) approaches in general offer different kinds of 
computational support for tracking learner behaviour, managing educational data, visualizing 
patterns, and providing rapid feedback to both educators and learners. This special section brings 
together a diverse range of learning analytics tools and techniques that can be deployed in the 
service of building 21 s century competencies. We introduce the research and development 
challenges, and introduce the research and practitioner papers accepted to this section, before 
concluding with some brief reflections on the collection and relevance of a complex systems 
perspective for framing this topic. 
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1 INTRODUCTION: 21 st CENTURY COMPETENCIES 

It has become a truism to assert that we live in an age of rapid change, in which technological 
and cultural disruptions create unprecedented complexity, turbulence, and uncertainty. A 
society's capacity to learn is central to its well-being, but economic, social, and technological 
turbulence places unprecedented pressure on citizens' capacity to deal with uncertainty and 
adapt to change. Citizens of all ages need increasingly to make sense of ambiguity with the loss 
of authority that used to surround educational, political, scientific, moral, religious, and other 
cultural institutions (Haste, 2001). In the school systems of technologically advanced nations, 
there are shocking figures around student disengagement, as young people struggle to see how 
what they learn connects with their technologically infused lives outside school (Buckingham 
Shum & Deakin Crick, 2012). In the world of work, employers complain increasingly that 
graduates from our school and university systems, while proficient at passing exams, have not 
developed the capacity to take responsibility for their own learning, and struggle when 
confronted by novel, real-world challenges (Haste, 2001). 
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We have experienced infrastructure revolutions before, but it is not overstating the case to see 
the explosion of the Internet, mobile computing, and now the Internet of devices as creating 
disruptions at an unprecedented pace. A growing body of scholarship in education (reviewed 
shortly), and on workplace futures (e.g., FYA, 2016; WEF, 2016) argues that educational systems 
— from primary school to higher degrees — must evolve beyond a sole focus on the mastery of 
knowledge and skills, in a predefined curriculum, assessed typically under controlled exam 
conditions. Such an approach assumes that the world at the time of learning will remain 
relatively stable, and that the required competencies can indeed be assessed in an exam. These 
assumptions, valid in an industrial era, no longer hold. 

A strategic educational response to a world of constant change is to focus explicitly on 
nurturing the skills and dispositions, assessed under authentic conditions, that equip learners to 
cope with novel, complex situations. Thus, even if we do not know what the future holds, we 
can be better equipped for the only thing we can be sure of — change. The qualities that 
learners need have thus been dubbed "21 st century" in nature — not because they were of no 
use before (although they may take novel forms today) — but because of their central 
importance in times of turbulence, in a jobs marketplace where routine cognitive work will be 
increasingly automated. 

2 DEFINING 21 st CENTURY COMPETENCIES 

What are these new qualities and competencies? There are many lists and taxonomies from 
numerous initiatives. A 2012 review by Pearson and the Canadian National Council on 
Measurement in Education identified critical thinking, creativity, collaboration, metacognition, 
and motivation as essential (Lai & Viering, 2012). A US National Research Council committee 
(Koenig, 2011) identified cognitive skills (Non-Routine Problem Solving, Systems Thinking and 
Critical Thinking), interpersonal skills (ranging from Active Listening, to Presentation Skills, to 
Conflict Resolution), and intrapersonal skills, which are personal qualities that equip a learner 
(broadly clustered under Adaptability and Self-Management/Self-Development). A large 
international joint academic/business project is under way, using a classification of Knowledge, 
Skills and Attitudes, Values and Ethics (ATC21S, http://www.atc21s.org). The first book from 
this project introduces the complex methodological and technological issues around 
assessment, with a taxonomy distilled from the literature as follows: Ways of Thinking, Ways of 
Working, Tools for Working and Living in the World (Griffin, McGaw, & Care, 2012). This survey 
included major European analyses such as the framework for key competencies (EU, 2006), and 
the OECD-CERI analyses of "new millennial learners" (OECD, 2012). Deakin Crick, Fluang, 
Ahmed-Shafi, and Goldspink (2015) report progress in a 15-year research program defining a 
multi-dimensional construct termed "learning power," focusing on the evidence for and 
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relationships between a set of malleable learning dispositions (rather than skills), namely: 
Mindful Agency, Sensemaking, Creativity, Curiosity, Belonging, Collaboration, Hope and 
Optimism, and Orientation to Learning (CLARA, 2016). 

3 LEARNING ANALYTICS FOR 21 st CENTURY COMPETENCIES? 

In their recent analysis of the field, Lai and Viering (2012) conclude: 

We recommend several practices for assessing 21 st century skills: incorporating multiple 
measures to permit triangulation of inferences; designing complex and/or challenging tasks; 
including open-ended and/or ill-structured tasks; using tasks that employ meaningful or 
authentic, real-world problem contexts; making student thinking and reasoning visible; and 
exploring innovative approaches that utilize new technology and psychometric models. 

This sets the challenging context for understanding the potential of LA approaches for the 
formative (and possibly summative) assessment of 21 st century competencies, which are 
important precisely because they need to be displayed in interpersonal, societal, and culturally 
valid contexts. By definition, the concept of assessing qualities that are lifelong — spanning the 
"arc of life" inside and beyond formal learning — demands new kinds of evidence and new 
forms of data literacy. Computational support for tracking, feeding back, and reflecting learning 
processes holds the promise that these qualities can be evidenced, at scale, in ways that have 
been impractical until now. 

Framed thus, the goal is to forge new links from the body of educational/learning sciences 
research — which typically clarifies the nature of the phenomena under question using 
representations and language for researchers — to documenting how data, algorithms, code, 
and user interfaces come together through coherent design in order to automate such analyses 
— providing actionable insight for the educators, students, and other stakeholders who 
constitute the learning system in question. 

Learning analytics of this sort is at an early stage of development, and this special section is 
intended to document examples of the current state of the art, and clarify primary challenges 
to advancing the field. Relevant work includes (but is not limited to) the established body of 
evidence on how learners' dispositions and mindsets impact engagement (Dweck, 2006; Deakin 
Crick et al. 2015; Krumm & Cheng, 2013), grounding efforts to develop practical formative 
assessment tools. Conscientiousness may be quantified through educational games (Shute & 
Ventura, 2013), while "epistemic frame analysis" can be used to design immersive simulation 
exercises with analytics (Shaffer et al., 2009). Language technologies grounded in learning 
sciences are beginning to illuminate the quality of interpersonal interaction in textual discourse 
(Rose & Tovares, 2015). 
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Quantifying these deeply personal qualities in order to feed back and strengthen them, without 
in the process reducing them to meaningless statistics, is at the heart of the learning analytics 
challenge. How does one gather data from a diversity of life contexts, as potential evidence of 
these new competencies? How do we translate theoretical constructs with integrity into 
algorithms to assess online behaviour? How can they be rendered for human interpretation, by 
whom, and with what training? Should such analytics be used primarily for formative 
assessment, or should we be aiming for summative grades? Who gets to design the analytics, 
and who gets to validate them? Do analytics of this sort raise new ethical dilemmas? 

4 INVITED TOPICS 

Contributions were invited to this special section to document and advance theory, design 
methodology, technology implementation, or evidence of impact, including but not limited to 
the following: 

• Analytics for higher order competencies such as critical thinking, curiosity, resilience, 
creativity, collaboration, sensemaking, self-regulation, reflection/meta-cognition, 
transdisciplinary thinking, or skillful improvisation 

• Theoretical arguments around the opportunities, or indeed the limits, for analytics in 
illuminating particular competencies 

• Principles and methodologies for combining complementary analytical approaches, 
including reflections on conventional educational assessment instruments, and 
computational approaches 

• Methodologies for validating analytics 

• Analytics for learning dispositions/mindsets/"non-cognitive" factors known to shape 
readiness to engage in learning 

• Analytics for different kinds of authentic assessment and inquiry-based learning 

• Technological challenges and opportunities for lifelong, life-wide learning analytics 
extending beyond formal educational contexts 

• Arguments regarding whether analytics could effect a shift in the assessment regimes, 
and associated pedagogies and epistemologies, promoted by conventional education 
policy 

• Analysis of the systemic organizational adoption issues for such analytics 
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• Visualization design for different user groups; in particular, to promote increasing 
learner self-awareness and capacity to take responsibility for one's learning 

5 OVERVIEW OF THE PAPERS 

We turn now to the contributions brought together in this special section, comprising five 
research papers and two practitioner papers. Starting with the research papers, in Towards the 
Discovery of Learner Metacognition from Reflective Writing, Gibson, Kitto, and Bruza argue the 
case for building learner capacities to engage in productive metacognition. First, they describe 
how this relates conceptually to reflection, defining a spectrum ranging from the unconscious 
inner-self through to the conscious external social self. This model motivates the use of 
reflective writing for formative assessment, and the model serves to guide exploratory 
computational analysis of undergraduate reflective writing. The authors recognize the early 
stage of the work as an analytics approach for automating the discovery of metacognitive 
activity in reflective writing, in order to provoke it further. 

Surprisingly, given the centrality of writing and online textual interaction in our educational 
systems, this is the only paper submitted that uses natural language processing (NLP). However, 
it represents an emerging category of learning analytics work that builds on the well- 
established NLP research community's tools and methods, but contextualizes them to learning. 
Within the learning analytics research community, we see evidence of growing interest in this 
(e.g., NLP-BEA, 2016; LAK-WA, 2016), while the commercial market expands as major publishers 
seek to address concerns over the quality of student writing (and educators' ability to grade), 
offering a range of summative and formative assessment products. 

In Revealing Opportunities for 21 st Century Learning: An Approach to Interpreting User Trace 
Log Data, Martin, Nacu, and Pinkard introduce a youth learning project designed to build digital 
media and online community engagement skills. They motivate a set of social learning analytics 
to code actions according to the relationships that hold between an actor and the recipient of 
online actions, mapping these actions to higher order constructs that match their desired 
outcomes: Creative Production, Self-Directed Learning, and Social Learning. They explore these 
results through the lens of individual learners, including cohort self-reports of identity, interest, 
and perceptions, and qualitative case studies of two students. 

This paper is distinctive in its "mixed methods" approach, combining student surveys and 
interviews, educator interviews and ethnographic methods to see if these illuminate 
quantitative log file analysis. The authors are appropriately cautious about what they can claim: 
while one cannot conclude, for instance, that a student is becoming more self-directed simply 
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because there is significant evidence of certain activity in the system logs, they triangulate their 
log data with their other data sources. 

In Understanding Learning and Learning Design in MOOCs: A Measurement-Based 
Interpretation, Milligan and Griffin operationalize a 21 st century competency associated with 
effective MOOC-based learning, which they term Crowd-Sourced Learning (C-SL) capability. This 
is defined as an "array of attitudes, beliefs, and understandings about learning that participants 
bring to a MOOC and which shape their behaviour and explain why individuals differ in their 
ability to generate higher order learning," comprising Epistemic Standpoint, Orientation to 
Teaching and Learning, and Regulation of Learning. They construct log file activity measures of 
C-SL's constituent sub-capabilities such as Critical Consumption, Production Orientation, and 
Risk Taking, enabling each MOOC learner to be assessed automatically on a scale from novice 
to expert. 

This paper demonstrates how a measurement science approach can draw on both the literature 
and educator's field knowledge of how students perform to inform the design of behavioural 
proxies in learning analytics. The result is an "operationalized assessment rubric" defining 
significant transitions from novice to expert: each cell in the rubric table has associated log file 
behaviours, which then permit evaluation of the impact of MOOC design iterations. 

In Practical Measurement and Productive Persistence: Strategies for Using Digital Learning 
System Data to Drive Improvement, Krumm et al. add another important perspective to the 
collection, grounding their work in Educational Improvement Science, an emerging 
methodology for designing research-inspired but intensely practical educational interventions, 
using a systems thinking approach. Their paper outlines the development of practical measures 
for a quality they define as Productive Persistence. Practical measurement refers to data 
collection and analysis approaches originating from improvement science; productive 
persistence refers to the combination of academic and social mindsets as well as learning 
behaviours that are important drivers of student success within their program. 

This paper is noteworthy in at least two respects. Firstly, we find the argument for 
improvement science to be a persuasive one, which we would encourage the learning analytics 
community to attend to: 1) it helps answer the question "where should we target our 
analytics?" because it provides a participatory methodology to work with educators to identify 
their most pressing challenges, and their key drivers; 2) analytics provide new ways to track 
those drivers and provide the rapid feedback loops critical to improvement cycles, answering 
"did we make a difference?" Second, part of this work focuses on Productive Persistence, a form 
of resilience sometimes referred to as a "non-cognitive" factor. Krumm et al. draw on the 
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concept of mindsets from Carol Dweck's work, also referred to as dispositions (Buckingham 
Shum & Deakin Crick, 2012), which have for some time been quantifiable from self-report 
survey measures (PERTS, 2016; Deakin Crick, Huang, Ahmed-Shafi and Goldspink, 2015; 
Tempelaar, Rienties, and Giesbers, 2015), but now we are beginning to see behavioural proxies 
(see also Shute & Ventura, 2013, who use video gaming to assess constructs such as Persistence 
and Perfectionism in children). 

In Analytics for Knowledge Creation: Towards Epistemic Agency and Design-Mode Thinking, 
Chen and Zhang report work arising from the long term learning sciences program developed 
by Scardamalia and Bereiter (e.g., 1991, 2014), into "Knowledge Building." Consequently, the 
primary argument is for the urgent need for the educational system to foster innovation 
(starting with children as young as 7 years in this paper). This motivates Chen and Zhang's focus 
on cultivating two higher order qualities, namely Epistemic Agency and Design-Mode Thinking. 
They document the process by which analytics for these qualities have been devised for 
collaborative tasks, and how the resulting patterns can be visualized for educators and students 
to provoke reflection. 

In contrast to the other papers, which take as input the data from relatively conventional 
learner interaction in learning platforms (e.g., writing; creating digital artifacts; watching 
movies; online discussion; solving maths problems), the analytics in this project are made 
possible because student activity is mediated by a structured, visual deliberation tool (cf. 
Andriessen, Baker and Suthers, 2003; Kirschner, Buckingham Shum and Carr, 2003). The 
resulting networks of hypertext nodes and links have semantics, sequences, and structural 
patterns that can be processed computationally far more easily than that required to analyze 
naturalistic online discourse (cf. Rose & Tovares, 2015), although the textual content of nodes is 
also analyzed. Students contribute by making higher order choices between a menu of 
discourse moves that serve as semantically meaningful units of activity for both learners and 
the machine, making it possible to implement novel analytics and visualizations. 

The section also includes two practitioner papers, written by and for practitioners (although we 
are certain academic researchers will also find them of interest). In the first. Tracking and 
Visualizing Student Effort: Evolution of a Practical Analytics Tool for Staff and Student 
Engagement, Robin Nagy documents an initiative to design an infrastructure (human work 
practices plus technology) to quantify and visualize a particular quality that his high school 
seeks to value, namely student effort. Nagy documents the iterative refinement of an approach 
deployed in successive versions over six years in secondary schools. This demonstrates how a 
student quality such as "effort" can be assessed by teachers in a practical way, and the insights 
that visual analytics can provide as a basis for productive dialogue among not only staff, but 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


12 


JOURNAL OF LEARNING ANALYTICS 


S 3LAR 

SOCIETY for LEARNING 
ANALYTICS RESEARCH 

(2016). Learning analytics for 21st century competencies. Journal of Learning Analytics, 3(2), 6-21. http://dx.doi.Org/10.18608/jla.2016.32.2 

with students who are intrigued to see their effort and attainment animated over time in 
relation to their peers, as the stimulus for a tutorial conversation. 

This practitioner paper adds to the section by documenting a participatory, systemic approach 
to analytics design. Nagy notes technical factors critical to the long-term sustainability of the 
tool, but also organizationally, the engagement and professional development of teachers is 
critical to embedding and sustaining novel analytics in the daily life of a busy high school. The 
approach demonstrates the power of quantifying and making visible a quality that otherwise 
remains intangible, and therefore hard to talk about or improve. The work shows the 
importance of iterating through successive classification schemes in order to whittle it down to 
the right balance of simplicity (teachers and students can grasp it) and expressiveness (it still 
makes useful distinctions between important constructs), which echoes the improvement 
science approach of Krumm et al.'s paper. As a practitioner paper, it relies not on the academic 
grounding of university research, but does provide statistical and qualitative evidence to back 
its claims. 

In the second practitioner paper, Marks Should Not Be the Focus of Assessment — But Flow Can 
Change Be Achieved? Darrall Thompson describes an assessment platform (REVIEW) that began 
as a university research prototype and, over a decade's refinement, launched as a product. The 
platform provides educators with tools to verify that their assessments are aligned with the 
stated learning outcomes, and an interface to integrate graduate attributes into their 
assessment rubrics for formative and summative feedback and grading ("Graduate Attributes" 
is the term used in the Australian higher education sector to refer to the transferable skills and 
dispositions that graduates should acquire in discipline-relevant ways). In REVIEW, students are 
able to engage in self-assessment prior to receiving their feedback, and then compare this with 
their actual grades and the cohort average. 

Chronologically, this work is the most mature of all the papers, in that Thompson is in a position 
to share over a decade's experience developing a tool, in a mainstream educational context, to 
quantify and visualize progress on 21 st century competencies. With the system in mainstream 
use in several universities, the codebase is enterprise grade, which is an encouraging stage to 
reach for a learning analytics platform. As a practitioner paper, in order to narrate the software 
design and its usage, this report devotes less space to formal validation of the approach, but 
refers the reader to the educational research on which it is grounded, and to which it has in 
turn contributed. The paper's overall message concerns the imperative to change assessment 
policy to value C21 qualities, given emerging evidence that REVIEW'S visual feedback is 
engaging for not only educators and students, but also for employers seeking robust evidence 
of students' readiness for the complexities of the workplace. 
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6 REFLECTIONS ON THIS COLLECTION OF PAPERS 

We have been struck by a number of points as we have worked with the authors and their 
papers. 

C21 learning analytics are at an early stage of maturity: In the research papers, there remains 
in our view quite a "gap" between the high-level constructs that authors wish to track and their 
operationalization in learner-activity log files. By "gap," we refer to the sophistication with 
which a construct has been defined, and in the revisions to their papers, we pushed the authors 
to make as clear as possible the claims they were making in this regard. We invite readers to 
assess these critically. It is perhaps no coincidence that the practitioner papers do not rely on 
sophisticated behavioural analytics, but rather gather their data from educator observation and 
student self-assessment. The role of technology in these tools is to aggregate quantitative data 
and display it in various summary forms, including visualizations, in order to provoke useful 
educator and student reflection. 

Absence of predictive modelling of "student success": Continuing the previous theme, when 
the goal is the cultivation of lifelong learning dispositions and skills, it is not by chance that we 
see no predictive modelling approaches: the goal is not course-completion per se, which is a 
common reference point for predictive modelling of "at-risk" students. C21 competencies could 
in principle assist in course completion — but only if the pedagogy and assessment regime has 
been designed to value rather than discourage such qualities. As we know, most courses do not 
because they are too hard to assess at scale using conventional instruments. The assessment 
task is of course critical. Authentic, transdisciplinary learning often places students in a complex 
scenario with no single correct answer. Such a learning environment becomes a complex 
adaptive system itself, in which educators and students are all learners. This is in stark contrast 
to closed learning systems with stable expertise (educators), and knowledge deficits (students), 
with known inputs and outputs — which defines a system far more amenable to user and 
process modelling, adaptive tutoring, and clearer grounds for declaring a student to be at risk, 
on track, or excelling. When it comes to building C21 analytics, much focus in these papers is on 
improving the feedback to learners so that they can have better conversations with peers and 
tutors, take increasing responsibility for their learning, and become mindful of such personal 
qualities as their sense of personal agency, openness to challenge, resilience in adversity, or 
reflective metacognition. 

Looking to the future, when C21 learning analytics have matured, and in concert, assessment 
design more explicitly values those competencies, teams will no doubt aim to develop "C21 
student success" predictive models. However, caution is advised. Consider, for example, if it 
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makes any sense to present an amber traffic signal to a student because s/he is not behaving as 
an archetype derived from the activities of previously creative or curious peers, tackling open- 
ended, authentic challenges. As a complex system, there are so many variables. The prospect of 
software acting autonomously and "adaptively," on the basis of a classifier using intrapersonal 
or interpersonal constructs, is fraught with ethical considerations — worthy of deep reflection 
and values-sensitive design. 

Strong presence of school-age projects: To date, most of the papers presented at the LAK 
conference and in this journal deal with students at university and beyond, so although not in 
any sense a statistically significant result, it may be noteworthy that four of the seven papers 
here are based on projects with children, a possible reflection of the higher visibility that C21 
discourse has in the school sector. The fact that it is beginning to be possible to nurture, and 
assess, such high level competencies in young people, in state schools operating under highly 
constrained national curricula, is in our view exciting and inspiring. Moreover, it raises the bar 
for higher education institutions, removing any excuses that undergraduates are not capable of 
such learning, or that educators cannot learn how to cultivate such qualities when they typically 
have greater control over curricula than in the school sector. Certainly within the innovation 
economies, we see employers — and the professional bodies sometimes blamed by academics 
for constraining how much they can innovate — placing as much emphasis on C21 
competencies as on the mastery of the (always evolving) professional knowledge base. 

A partial snapshot of C21 learning analytics: These papers provide a snapshot of the state of 
the art, providing helpful landmarks as we map this new territory. However, this collection is by 
no means complete in its coverage of relevant work nor in tackling the challenges in the Call for 
Papers. Approaches one would also want to consider include educational gaming analytics (e.g., 
Shute and Ventura, 2013; Shaffer, 2013), multimodal analytics on embodied presentation skills 
(e.g., Echeverria, Avendano, Chiluiza, Vasquez, & Ochoa, 2014) and face-to-face collaboration 
(Martinez-Maldonado et al., 2016), computer-supported collaborative problem solving tests 
(Griffin et al., 2012), self-regulation (Roll & Winne, 2015), social learning analytics (e.g.. Tan, 
Yang, Koh, & Jonathan, 2016), and "quantified self" personal data (e.g., Eynon, 2015). 

7 A COMPLEX SYSTEMS PERSPECTIVE 

Complex systems perspectives are beginning to be applied to learning analytics to help make 
sense of the organizational dynamics of introducing analytics (Macfadyen, Dawson, Pardo, & 
Gasevic, 2014; Colvin, et al. 2016). We also note important work in the Science and Technology 
Studies community on information and knowledge infrastructures (Bowker & Star, 1999; 
Edwards et al. 2013), and the growing body of work on big data, ethics, and society (e.g.. 
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CBDES, http://bdes.datasociety.net/). These programs expand the boundaries of "the system" 
and the stakeholders we should consider in learning analytics, drawing attention to the ways in 
which power is (re)distributed by such infrastructures, and the many levels at which values are 
baked into them, with the risk that they become invisible, and unaccountable (Buckingham 
Shum, 2016). 

We propose that a transdisciplinary, complex systems perspective is particularly well suited to 
the distinctive challenge of conceiving learning analytics for C21 qualities. There are two 
fundamental challenges implicit in the task we have set, which are evident in the projects in this 
collection. First is the transdisciplinary nature of the work and the concomitant requirement for 
rigour and expertise in, at least, pedagogy, learning sciences, computation, technology, and 
assessment. Second is the engaged nature of the work in which students, teachers, and leaders 
as users of learning systems are critical stakeholders alongside academics and technologists. 
Engaging with this complexity is inevitable, and understanding it requires us to transcend a 
single, discipline-based perspective. 

Drawing on work in systems thinking and complexity (Blockley, 2010), we identify some 
characteristics of complex systems that may help us to make sense of the challenge. In 
conditions of complexity and uncertainty, the question of the purpose becomes paramount and 
provides a powerful lens through which to identify system boundaries and the system's social, 
organizational, and technical architecture — the processes that matter. Only by identifying the 
purpose of the system can we hope to evaluate it. The purpose of learning analytics to 
formatively support the development of C21 competencies in students means that the overall 
desired outcome is students with a set of capabilities, addressed above, which by definition 
include the full range of "human interests" — empirical, analytical, hermeneutical, and 
emancipatory (Joldersma & Deakin Crick, 2010) together with their distinctive "rationalities" 
and data forms. This means at the very least, as evidenced in these studies, that we must learn 
how to responsibly capture, analyze, and use rich data about student learning, including 
attitudes, values and dispositions, narrative, purpose, and identity — as well as knowledge 
generating processes and the more familiar learning outcome measures (Deakin Crick, 2012). 
This includes making reliable links "from clicks to constructs" — the new version of making 
inferences from behaviour to constructs. 

A learning analytics "system" operates at several levels — users who may be students or 
teachers, teachers making pedagogical decisions about groups, leaders making decisions about 
improvement strategies for the organization, policy makers, and researchers. The studies in this 
collection address learning analytics tuned to formative learning and decision making at several 
of these levels. 
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Another characteristic of complex systems is feedback loops that influence the system 
processes for better or for worse. Each study in some way captures and feeds back data 
presumed to be helpful for formative change and self-directed learning or teaching. The 
affordance of technology to generate feedback in real or rapid time is a key aspect of learning 
analytics that should not be underestimated when it comes to developing wider C21 
competences, because it abolishes the lifecycle gap between traditional research and practice. 
It also creates the demand for "practical" measures that are both trustworthy and useful in the 
"messy" and "time poor" world of the classroom or workplace. Linked to this — and evident 
also in these studies is the power of technology to re-present knowledge in a variety of forms, 
particularly visual, that convey meaning in very different ways from traditional reporting. 
Chen's work in the "promising ideas tool," for example, goes beyond simply re-presenting 
knowledge in novel ways to something more akin to the creation of a "virtual zone of proximal 
development" (Vygotsky, 1978, p. 159) that can be inhabited by the learner, rendering the 
technology more like a "psycho-prosthetic limb" — a knowledge generating tool. 

The processes that learning analytics aim to serve and enhance in the development of C21 
competencies are more problematic because there is not a consensus about how they fit 
together, how they operate in learning and teaching, and how they relate to the traditional 
competencies and skills measured in education. They are sometimes even defined by what they 
are not — i.e., non-cognitive skills — which carries an implicit value judgment. There are many 
candidate processes discussed in this introduction and present, too, in all of the studies. What 
is core to most of them is that, to achieve their purpose, they need to be driven by the learner's 
agency and choice. Reflection, for example, is used by students for developing deeper 
understanding and scaffolding meta-cognition. Self-regulation, self-efficacy, and productive 
persistence, by definition, require a person to be efficacious and regulate the self. Creative 
knowledge building is about producing something original in an authentic context — to which 
student choice (and thus agency and purpose) are key. Furthermore, this move towards the 
learner as a "self-organizing system" brings with it a similar requirement at each level of the 
system. Given learning analytics for C21 competencies, teachers or tutors can no longer deliver 
a pre-determined script or curriculum — they are required to received, collate, analyze, and 
respond to complex data about real learners in close to real time and make pedagogical 
decisions in situ, in an on-going cycle of improvement. They function more like "learning 
designers" (Deakin Crick and Goldspink, 2014) where they attend to their purpose, the context 
in which they operate, and the needs of students to synergize these into next best actions. They 
manage what "emerges" rather than simply "delivering expert knowledge." 

This capacity of learning analytics to enable self-organized decision-making at every level has 
implications for leaders, policy makers, and researchers too. How can we align all stakeholders 
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around a shared improvement aim whilst also enabling responsibility, authority, and 
accountability to be aligned appropriately at each level? What sort of leadership analytics and 
dashboards can support this challenge? How can governments re-think accountability 
frameworks to support this? Improvement science is an approach that does justice to the 
micro-level of learning at the same time as an overall macro level improvement aim, and 
learning analytics of the sort described in this special section are a powerful resource to 
support this approach because they allow feedback at different levels on processes of learning 
to the people responsible for change. 

Learning analytics is an emerging field powered by the paradigm shifts of the information age. 
Pedagogy and learning that produce students capable of thriving in conditions of complexity, 
risk, and challenge is also an emergent field, still struggling to find its way. A significant program 
of work is opening up, and we hope that this special section assists in mapping the known 
territory, as well as where to explore next. 
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